Submissions & Scores
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
BART-base
mT5_base
mT5_large
mT5_small
mT5_xl
NUIG-DSI
POINTER
t5-small
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
TGen_lemma-tag+RNNLM
TGen+RNNLM
TGen
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
BART-base
mT5_base
mT5_large
mT5_small
mT5_xl
NUIG-DSI
t5-small
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
FB_NLG
NUIG-DSI
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
t5-small
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
FB_NLG
NUIG-DSI
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
FB_NLG
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
BART-base
mT5_base
mT5_large
mT5_small
mT5_xl
SimpleNER_baseline
SimpleNER
t5-small
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
BART-base
mT5_base
mT5_large
mT5_small
mT5_xl
SimpleNER_baseline
SimpleNER
t5-small
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
ByT5-base (Baseline)
ByT5-large (Baseline)
ByT5-small (Baseline)
ByT5-xl (Baseline)
PEGASUS
T5-base (Baseline)
T5-large (Baseline)
T5-small (Baseline)
T5-xl (Baseline)
mT5_base
mT5_large
mT5_small
mT5_xl
Measures
Output Length
Vocabulary Size
Bigram Vocabulary Size
MSTTR
Distinct-1
Distinct-2
Unique-1
Unique-2
Entropy-1
Entropy-2
Bigram Conditional Entropy
NUBIA
ROUGE-1
ROUGE-2
ROUGE-L
BLEU
Meteor
SARI
NIST
BERTScore
BLEURT
Visualization
Table
Results:, Measures:
mean_pred_length | vocab_size-1 | vocab_size-2 | cond_entropy-2 | distinct-1 | distinct-2 | entropy-1 | entropy-2 | msttr-100 | unique-1 | unique-2 | nubia | bleu | meteor | nist | rouge1 | rouge2 | rougeL | sari | bertscore | bleurt | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| data2text | common gen test | |||||||||||||||||||||
| dart test | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | T5-xl (Baseline) | |||||||||||
| e2e nlg test | T5-base (Baseline) T5-large (Baseline) T5-xl (Baseline) T5-small (Baseline) mT5_xl | ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) T5-xl (Baseline) | ByT5-base (Baseline) T5-xl (Baseline) ByT5-small (Baseline) T5-large (Baseline) ByT5-xl (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) T5-xl (Baseline) ByT5-xl (Baseline) ByT5-large (Baseline) | ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) T5-xl (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) ByT5-large (Baseline) T5-xl (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) ByT5-large (Baseline) T5-xl (Baseline) | ByT5-base (Baseline) T5-xl (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) T5-large (Baseline) | T5-xl (Baseline) ByT5-base (Baseline) T5-large (Baseline) T5-base (Baseline) ByT5-small (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) T5-xl (Baseline) ByT5-large (Baseline) ByT5-xl (Baseline) | ByT5-small (Baseline) ByT5-base (Baseline) T5-xl (Baseline) ByT5-xl (Baseline) T5-large (Baseline) | mT5_large mT5_base FB_NLG mT5_xl T5-small (Baseline) | mT5_large mT5_xl mT5_base T5-small (Baseline) mT5_small | mT5_large mT5_xl mT5_base FB_NLG T5-small (Baseline) | mT5_large FB_NLG mT5_xl mT5_base T5-small (Baseline) | FB_NLG mT5_base mT5_large mT5_xl mT5_small | mT5_base mT5_large NUIG-DSI FB_NLG mT5_small | NUIG-DSI mT5_base mT5_small mT5_large mT5_xl | mT5_large mT5_base FB_NLG mT5_xl mT5_small | mT5_base mT5_large mT5_xl mT5_small FB_NLG | ||
| totto test | T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-base (Baseline) | T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-base (Baseline) | T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-base (Baseline) | T5-xl (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) T5-large (Baseline) ByT5-xl (Baseline) | t5-small ByT5-xl (Baseline) ByT5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) | ByT5-small (Baseline) ByT5-base (Baseline) ByT5-xl (Baseline) T5-large (Baseline) T5-xl (Baseline) | T5-base (Baseline) ByT5-xl (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-large (Baseline) | T5-xl (Baseline) ByT5-small (Baseline) T5-large (Baseline) ByT5-base (Baseline) T5-base (Baseline) | ByT5-xl (Baseline) ByT5-base (Baseline) t5-small T5-base (Baseline) ByT5-large (Baseline) | T5-xl (Baseline) ByT5-base (Baseline) T5-large (Baseline) ByT5-xl (Baseline) ByT5-small (Baseline) | T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-base (Baseline) | mT5_xl T5-xl (Baseline) T5-base (Baseline) T5-large (Baseline) ByT5-xl (Baseline) | T5-base (Baseline) T5-xl (Baseline) ByT5-xl (Baseline) mT5_xl T5-large (Baseline) | mT5_xl T5-xl (Baseline) T5-base (Baseline) mT5_large T5-large (Baseline) | T5-base (Baseline) mT5_xl ByT5-xl (Baseline) T5-xl (Baseline) T5-large (Baseline) | T5-base (Baseline) T5-xl (Baseline) T5-large (Baseline) ByT5-xl (Baseline) mT5_xl | T5-base (Baseline) ByT5-xl (Baseline) T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) | T5-base (Baseline) ByT5-xl (Baseline) T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) | ByT5-xl (Baseline) T5-base (Baseline) mT5_xl T5-xl (Baseline) mT5_large | T5-base (Baseline) mT5_xl mT5_large ByT5-xl (Baseline) T5-xl (Baseline) | ||
| cs restaurants test | T5-large (Baseline) T5-xl (Baseline) T5-base (Baseline) T5-small (Baseline) mT5_large | T5-large (Baseline) T5-xl (Baseline) T5-small (Baseline) ByT5-xl (Baseline) T5-base (Baseline) | T5-large (Baseline) T5-xl (Baseline) T5-small (Baseline) ByT5-large (Baseline) ByT5-xl (Baseline) | T5-small (Baseline) T5-xl (Baseline) T5-large (Baseline) T5-base (Baseline) ByT5-large (Baseline) | ByT5-small (Baseline) mT5_xl ByT5-base (Baseline) ByT5-xl (Baseline) ByT5-large (Baseline) | ByT5-small (Baseline) ByT5-large (Baseline) mT5_xl ByT5-xl (Baseline) ByT5-base (Baseline) | ByT5-xl (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) mT5_xl ByT5-base (Baseline) | T5-small (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) mT5_xl | ByT5-small (Baseline) ByT5-xl (Baseline) ByT5-base (Baseline) mT5_xl ByT5-large (Baseline) | T5-small (Baseline) T5-xl (Baseline) T5-large (Baseline) T5-base (Baseline) ByT5-xl (Baseline) | ByT5-large (Baseline) ByT5-xl (Baseline) T5-xl (Baseline) T5-large (Baseline) T5-small (Baseline) | TGen_lemma-tag+RNNLM TGen TGen+RNNLM ByT5-base (Baseline) ByT5-small (Baseline) | TGen_lemma-tag+RNNLM TGen+RNNLM TGen mT5_base mT5_xl | TGen TGen+RNNLM mT5_large mT5_base TGen_lemma-tag+RNNLM | TGen_lemma-tag+RNNLM TGen TGen+RNNLM mT5_base mT5_xl | TGen_lemma-tag+RNNLM TGen TGen+RNNLM ByT5-base (Baseline) mT5_base | TGen_lemma-tag+RNNLM TGen TGen+RNNLM mT5_base mT5_large | TGen_lemma-tag+RNNLM TGen TGen+RNNLM mT5_base ByT5-base (Baseline) | TGen_lemma-tag+RNNLM mT5_large mT5_base TGen TGen+RNNLM | TGen TGen_lemma-tag+RNNLM TGen+RNNLM mT5_large mT5_base | ||
| web nlg en test | mT5_small T5-small (Baseline) T5-base (Baseline) mT5_base mT5_large | T5-small (Baseline) T5-large (Baseline) T5-base (Baseline) mT5_small ByT5-base (Baseline) | T5-large (Baseline) T5-base (Baseline) T5-small (Baseline) T5-xl (Baseline) mT5_small | T5-large (Baseline) T5-base (Baseline) T5-small (Baseline) mT5_small T5-xl (Baseline) | ByT5-base (Baseline) ByT5-large (Baseline) ByT5-xl (Baseline) ByT5-small (Baseline) T5-large (Baseline) | ByT5-xl (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) T5-large (Baseline) | ByT5-xl (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) T5-xl (Baseline) | T5-large (Baseline) T5-base (Baseline) T5-xl (Baseline) ByT5-large (Baseline) ByT5-xl (Baseline) | ByT5-xl (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) NUIG-DSI | T5-small (Baseline) T5-large (Baseline) T5-base (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) | T5-large (Baseline) T5-base (Baseline) T5-small (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) | FB_NLG mT5_xl T5-xl (Baseline) mT5_large mT5_base | FB_NLG mT5_xl T5-xl (Baseline) mT5_large mT5_base | FB_NLG mT5_xl T5-xl (Baseline) mT5_large mT5_base | FB_NLG mT5_xl T5-xl (Baseline) mT5_large mT5_base | FB_NLG T5-xl (Baseline) mT5_xl mT5_large ByT5-base (Baseline) | FB_NLG mT5_xl T5-xl (Baseline) ByT5-base (Baseline) ByT5-xl (Baseline) | FB_NLG mT5_xl ByT5-base (Baseline) T5-xl (Baseline) ByT5-xl (Baseline) | FB_NLG mT5_xl T5-xl (Baseline) mT5_large ByT5-base (Baseline) | FB_NLG mT5_xl T5-xl (Baseline) mT5_large NUIG-DSI | ||
| web nlg ru test | T5-small (Baseline) T5-large (Baseline) T5-xl (Baseline) T5-base (Baseline) mT5_base | mT5_large mT5_base mT5_xl ByT5-large (Baseline) ByT5-base (Baseline) | T5-xl (Baseline) T5-large (Baseline) mT5_large mT5_base mT5_xl | T5-xl (Baseline) T5-large (Baseline) T5-base (Baseline) T5-small (Baseline) ByT5-xl (Baseline) | ByT5-large (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) ByT5-xl (Baseline) mT5_large | ByT5-large (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) mT5_large mT5_base | ByT5-large (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) mT5_large mT5_xl | mT5_large ByT5-large (Baseline) mT5_base ByT5-small (Baseline) mT5_xl | ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) mT5_large mT5_xl | ByT5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-xl (Baseline) mT5_large | ByT5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-xl (Baseline) T5-large (Baseline) | mT5_xl mT5_base mT5_large mT5_small ByT5-base (Baseline) | mT5_xl mT5_large mT5_base mT5_small ByT5-base (Baseline) | mT5_large mT5_xl mT5_base mT5_small ByT5-base (Baseline) | mT5_xl mT5_large mT5_base mT5_small ByT5-base (Baseline) | mT5_xl mT5_large mT5_base T5-xl (Baseline) T5-large (Baseline) | mT5_xl mT5_base mT5_large T5-xl (Baseline) mT5_small | mT5_xl mT5_large mT5_base T5-xl (Baseline) T5-base (Baseline) | mT5_large mT5_xl mT5_base mT5_small ByT5-base (Baseline) | mT5_large mT5_base mT5_xl mT5_small ByT5-base (Baseline) | ||
| summarization | mlsum de test | mT5_xl T5-xl (Baseline) mT5_large mT5_base T5-large (Baseline) | T5-xl (Baseline) mT5_xl mT5_base mT5_small mT5_large | mT5_xl T5-xl (Baseline) mT5_base mT5_large mT5_small | mT5_large mT5_xl mT5_base T5-xl (Baseline) mT5_small | ByT5-small (Baseline) ByT5-large (Baseline) ByT5-base (Baseline) ByT5-xl (Baseline) T5-base (Baseline) | mT5_base mT5_small ByT5-small (Baseline) T5-xl (Baseline) mT5_large | T5-xl (Baseline) mT5_xl mT5_base mT5_small mT5_large | T5-xl (Baseline) mT5_xl mT5_base mT5_large mT5_small | mT5_base T5-xl (Baseline) mT5_large mT5_xl mT5_small | T5-xl (Baseline) mT5_xl mT5_base mT5_small mT5_large | mT5_xl T5-xl (Baseline) mT5_base mT5_large mT5_small | T5-xl (Baseline) mT5_xl mT5_large mT5_base T5-base (Baseline) | T5-base (Baseline) mT5_small mT5_base mT5_large T5-xl (Baseline) | T5-xl (Baseline) mT5_xl mT5_large mT5_base mT5_small | T5-base (Baseline) mT5_base mT5_large mT5_small T5-xl (Baseline) | T5-xl (Baseline) mT5_xl mT5_large mT5_base T5-base (Baseline) | T5-xl (Baseline) mT5_xl mT5_large mT5_base T5-base (Baseline) | T5-xl (Baseline) mT5_xl mT5_large mT5_base T5-base (Baseline) | T5-xl (Baseline) mT5_xl mT5_large mT5_base T5-base (Baseline) | mT5_xl mT5_large T5-xl (Baseline) mT5_base T5-base (Baseline) | |
| mlsum es test | ByT5-xl (Baseline) T5-base (Baseline) mT5_base T5-small (Baseline) T5-large (Baseline) | mT5_base mT5_xl T5-base (Baseline) ByT5-base (Baseline) mT5_small | mT5_base mT5_xl T5-base (Baseline) mT5_large mT5_small | mT5_large mT5_xl mT5_base mT5_small ByT5-base (Baseline) | ByT5-large (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) mT5_base mT5_small | ByT5-small (Baseline) ByT5-large (Baseline) ByT5-base (Baseline) mT5_base mT5_xl | mT5_base ByT5-small (Baseline) mT5_xl ByT5-base (Baseline) mT5_large | mT5_base mT5_xl mT5_large ByT5-small (Baseline) ByT5-base (Baseline) | mT5_large mT5_base mT5_xl ByT5-base (Baseline) ByT5-large (Baseline) | mT5_base mT5_xl ByT5-base (Baseline) T5-base (Baseline) ByT5-small (Baseline) | mT5_base mT5_xl T5-base (Baseline) mT5_large mT5_small | mT5_xl ByT5-base (Baseline) mT5_large ByT5-small (Baseline) ByT5-xl (Baseline) | mT5_xl mT5_base mT5_large ByT5-base (Baseline) mT5_small | mT5_xl mT5_large mT5_base ByT5-base (Baseline) ByT5-small (Baseline) | mT5_xl mT5_large mT5_base ByT5-base (Baseline) mT5_small | mT5_xl mT5_large T5-xl (Baseline) ByT5-base (Baseline) mT5_base | mT5_xl T5-xl (Baseline) mT5_large ByT5-base (Baseline) mT5_base | mT5_xl T5-xl (Baseline) mT5_large ByT5-base (Baseline) mT5_base | mT5_xl mT5_large mT5_base ByT5-base (Baseline) T5-xl (Baseline) | mT5_xl mT5_large ByT5-base (Baseline) mT5_base mT5_small | ||
| xsum test | ByT5-base (Baseline) mT5_base T5-xl (Baseline) T5-base (Baseline) ByT5-small (Baseline) | T5-xl (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) T5-large (Baseline) T5-base (Baseline) | T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-base (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) T5-xl (Baseline) T5-large (Baseline) | ByT5-large (Baseline) T5-xl (Baseline) mT5_xl ByT5-xl (Baseline) PEGASUS | T5-xl (Baseline) T5-large (Baseline) ByT5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) | T5-xl (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) T5-large (Baseline) | T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) | T5-xl (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) T5-large (Baseline) PEGASUS | T5-xl (Baseline) ByT5-large (Baseline) PEGASUS ByT5-base (Baseline) T5-large (Baseline) | T5-xl (Baseline) T5-large (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) | T5-xl (Baseline) mT5_xl T5-large (Baseline) ByT5-xl (Baseline) T5-base (Baseline) | T5-xl (Baseline) T5-large (Baseline) mT5_xl ByT5-xl (Baseline) T5-base (Baseline) | T5-xl (Baseline) mT5_xl T5-large (Baseline) ByT5-xl (Baseline) T5-base (Baseline) | T5-xl (Baseline) mT5_xl ByT5-xl (Baseline) T5-large (Baseline) T5-base (Baseline) | T5-xl (Baseline) mT5_xl ByT5-xl (Baseline) T5-large (Baseline) T5-base (Baseline) | T5-xl (Baseline) mT5_xl ByT5-xl (Baseline) T5-large (Baseline) T5-base (Baseline) | T5-xl (Baseline) mT5_xl ByT5-xl (Baseline) T5-large (Baseline) T5-base (Baseline) | T5-xl (Baseline) mT5_xl T5-large (Baseline) ByT5-xl (Baseline) T5-base (Baseline) | T5-xl (Baseline) mT5_xl T5-large (Baseline) ByT5-xl (Baseline) T5-base (Baseline) | ||
| wiki lingua turkish tr test | T5-large (Baseline) T5-small (Baseline) T5-xl (Baseline) T5-base (Baseline) mT5_small | T5-xl (Baseline) T5-base (Baseline) mT5_large mT5_base mT5_small | T5-xl (Baseline) T5-base (Baseline) mT5_large mT5_xl mT5_small | T5-xl (Baseline) mT5_xl T5-base (Baseline) mT5_large ByT5-small (Baseline) | ByT5-xl (Baseline) ByT5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) mT5_xl | ByT5-xl (Baseline) ByT5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) mT5_xl | ByT5-xl (Baseline) T5-xl (Baseline) ByT5-large (Baseline) T5-base (Baseline) ByT5-base (Baseline) | T5-xl (Baseline) T5-base (Baseline) mT5_xl mT5_large mT5_base | ByT5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-xl (Baseline) T5-base (Baseline) | ByT5-xl (Baseline) T5-xl (Baseline) T5-base (Baseline) mT5_large ByT5-large (Baseline) | T5-xl (Baseline) T5-base (Baseline) mT5_large mT5_xl mT5_base | mT5_xl mT5_large ByT5-base (Baseline) mT5_base T5-xl (Baseline) | mT5_xl mT5_large mT5_base T5-xl (Baseline) mT5_small | mT5_xl mT5_large mT5_base T5-xl (Baseline) mT5_small | mT5_xl mT5_large mT5_base T5-xl (Baseline) mT5_small | mT5_xl ByT5-base (Baseline) mT5_large ByT5-small (Baseline) mT5_base | mT5_xl ByT5-base (Baseline) mT5_large ByT5-small (Baseline) mT5_base | mT5_xl ByT5-base (Baseline) ByT5-small (Baseline) mT5_large ByT5-large (Baseline) | mT5_xl mT5_large ByT5-base (Baseline) mT5_base ByT5-small (Baseline) | mT5_xl ByT5-base (Baseline) mT5_large ByT5-small (Baseline) ByT5-large (Baseline) | mT5_xl mT5_large mT5_base ByT5-base (Baseline) mT5_small | |
| wiki lingua vietnamese vi test | T5-small (Baseline) T5-large (Baseline) mT5_small T5-xl (Baseline) mT5_large | T5-xl (Baseline) mT5_xl ByT5-base (Baseline) ByT5-large (Baseline) mT5_large | T5-xl (Baseline) mT5_xl ByT5-base (Baseline) mT5_large ByT5-small (Baseline) | ByT5-small (Baseline) ByT5-base (Baseline) mT5_xl T5-xl (Baseline) ByT5-large (Baseline) | ByT5-base (Baseline) ByT5-large (Baseline) ByT5-xl (Baseline) ByT5-small (Baseline) T5-xl (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) T5-xl (Baseline) mT5_xl | ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) T5-xl (Baseline) mT5_xl | T5-xl (Baseline) mT5_xl ByT5-base (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) | ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) T5-xl (Baseline) mT5_xl | T5-xl (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) ByT5-xl (Baseline) mT5_xl | mT5_xl T5-xl (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) | mT5_xl mT5_small mT5_large mT5_base T5-base (Baseline) | mT5_xl mT5_large T5-xl (Baseline) ByT5-xl (Baseline) ByT5-base (Baseline) | mT5_xl mT5_large T5-xl (Baseline) ByT5-xl (Baseline) ByT5-base (Baseline) | mT5_xl mT5_large T5-xl (Baseline) mT5_base ByT5-base (Baseline) | mT5_xl ByT5-xl (Baseline) mT5_large ByT5-base (Baseline) ByT5-small (Baseline) | mT5_xl ByT5-xl (Baseline) ByT5-base (Baseline) mT5_large ByT5-small (Baseline) | mT5_xl ByT5-xl (Baseline) mT5_large ByT5-base (Baseline) ByT5-small (Baseline) | mT5_xl ByT5-xl (Baseline) mT5_large ByT5-base (Baseline) T5-xl (Baseline) | mT5_xl ByT5-xl (Baseline) mT5_large ByT5-base (Baseline) ByT5-small (Baseline) | mT5_xl mT5_large ByT5-xl (Baseline) T5-xl (Baseline) mT5_base | |
| wiki lingua spanish es test | mT5_small mT5_xl T5-xl (Baseline) mT5_base T5-large (Baseline) | ByT5-xl (Baseline) mT5_xl T5-xl (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) | mT5_xl T5-xl (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-large (Baseline) | mT5_xl T5-xl (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) | ByT5-xl (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) T5-xl (Baseline) | ByT5-xl (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) mT5_xl T5-xl (Baseline) | ByT5-xl (Baseline) ByT5-base (Baseline) mT5_xl ByT5-small (Baseline) T5-xl (Baseline) | mT5_xl T5-xl (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) mT5_xl T5-xl (Baseline) ByT5-xl (Baseline) | ByT5-xl (Baseline) ByT5-base (Baseline) T5-xl (Baseline) mT5_xl ByT5-small (Baseline) | mT5_xl T5-xl (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) T5-large (Baseline) | T5-xl (Baseline) mT5_xl mT5_small mT5_large mT5_base | T5-xl (Baseline) mT5_xl T5-large (Baseline) mT5_large ByT5-base (Baseline) | T5-xl (Baseline) mT5_xl T5-large (Baseline) mT5_large ByT5-base (Baseline) | mT5_xl T5-xl (Baseline) T5-large (Baseline) mT5_base mT5_large | T5-xl (Baseline) mT5_xl T5-large (Baseline) ByT5-base (Baseline) ByT5-small (Baseline) | T5-xl (Baseline) mT5_xl ByT5-base (Baseline) ByT5-small (Baseline) T5-large (Baseline) | T5-xl (Baseline) mT5_xl mT5_large ByT5-base (Baseline) T5-large (Baseline) | T5-xl (Baseline) mT5_xl T5-large (Baseline) mT5_large ByT5-base (Baseline) | T5-xl (Baseline) mT5_xl mT5_large T5-large (Baseline) ByT5-base (Baseline) | T5-xl (Baseline) mT5_xl mT5_large T5-large (Baseline) mT5_base | |
| wiki lingua russian ru test | T5-small (Baseline) T5-large (Baseline) mT5_xl mT5_large T5-xl (Baseline) | mT5_xl T5-xl (Baseline) ByT5-large (Baseline) ByT5-xl (Baseline) ByT5-small (Baseline) | mT5_xl T5-xl (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) mT5_large | mT5_xl T5-xl (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) | ByT5-large (Baseline) ByT5-xl (Baseline) ByT5-small (Baseline) ByT5-base (Baseline) T5-xl (Baseline) | ByT5-large (Baseline) ByT5-small (Baseline) mT5_xl T5-xl (Baseline) ByT5-xl (Baseline) | mT5_xl ByT5-large (Baseline) ByT5-small (Baseline) T5-xl (Baseline) ByT5-xl (Baseline) | mT5_xl T5-xl (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) | ByT5-large (Baseline) mT5_xl T5-xl (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) | ByT5-large (Baseline) T5-xl (Baseline) mT5_xl ByT5-xl (Baseline) ByT5-small (Baseline) | mT5_xl T5-xl (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) ByT5-xl (Baseline) | mT5_xl mT5_small mT5_large T5-xl (Baseline) mT5_base | mT5_xl T5-xl (Baseline) mT5_large ByT5-xl (Baseline) ByT5-base (Baseline) | mT5_xl T5-xl (Baseline) mT5_large ByT5-base (Baseline) ByT5-xl (Baseline) | mT5_xl T5-xl (Baseline) mT5_large T5-large (Baseline) mT5_small | mT5_xl T5-xl (Baseline) mT5_large ByT5-base (Baseline) ByT5-xl (Baseline) | mT5_xl T5-xl (Baseline) mT5_large ByT5-xl (Baseline) ByT5-base (Baseline) | mT5_xl T5-xl (Baseline) mT5_large ByT5-base (Baseline) ByT5-xl (Baseline) | mT5_xl T5-xl (Baseline) mT5_large ByT5-base (Baseline) ByT5-xl (Baseline) | mT5_xl T5-xl (Baseline) mT5_large ByT5-base (Baseline) ByT5-xl (Baseline) | mT5_xl T5-xl (Baseline) mT5_large mT5_base ByT5-base (Baseline) | |
| dialog | schema guided dialog test | FB_NLG T5-xl (Baseline) mT5_large T5-large (Baseline) T5-base (Baseline) | T5-xl (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) ByT5-small (Baseline) T5-large (Baseline) | ByT5-small (Baseline) T5-xl (Baseline) ByT5-base (Baseline) ByT5-large (Baseline) T5-large (Baseline) | ByT5-small (Baseline) ByT5-base (Baseline) T5-xl (Baseline) ByT5-large (Baseline) T5-large (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) T5-xl (Baseline) T5-large (Baseline) | ByT5-small (Baseline) ByT5-base (Baseline) T5-xl (Baseline) ByT5-large (Baseline) T5-large (Baseline) | ByT5-base (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) T5-xl (Baseline) mT5_large | ByT5-base (Baseline) ByT5-small (Baseline) T5-xl (Baseline) ByT5-large (Baseline) T5-large (Baseline) | ByT5-base (Baseline) T5-xl (Baseline) ByT5-small (Baseline) ByT5-large (Baseline) mT5_large | ByT5-base (Baseline) ByT5-large (Baseline) T5-xl (Baseline) T5-large (Baseline) ByT5-small (Baseline) | ByT5-small (Baseline) ByT5-base (Baseline) T5-xl (Baseline) ByT5-large (Baseline) T5-large (Baseline) | FB_NLG ByT5-xl (Baseline) mT5_xl T5-base (Baseline) mT5_base | FB_NLG mT5_xl T5-base (Baseline) ByT5-xl (Baseline) T5-small (Baseline) | FB_NLG mT5_xl ByT5-xl (Baseline) T5-base (Baseline) T5-xl (Baseline) | FB_NLG mT5_xl ByT5-xl (Baseline) T5-base (Baseline) ByT5-base (Baseline) | FB_NLG mT5_xl ByT5-xl (Baseline) T5-base (Baseline) T5-small (Baseline) | FB_NLG mT5_xl ByT5-xl (Baseline) T5-base (Baseline) T5-small (Baseline) | FB_NLG ByT5-xl (Baseline) mT5_xl T5-base (Baseline) T5-small (Baseline) | FB_NLG ByT5-xl (Baseline) mT5_xl T5-base (Baseline) T5-small (Baseline) | FB_NLG ByT5-xl (Baseline) mT5_xl T5-base (Baseline) T5-small (Baseline) | |
| simplification | wiki auto asset turk test asset | SimpleNER t5-small mT5_large mT5_base mT5_small | t5-small mT5_small mT5_large mT5_base BART-base | t5-small mT5_small mT5_large mT5_base mT5_xl | SimpleNER t5-small SimpleNER_baseline BART-base mT5_xl | ByT5-small (Baseline) ByT5-base (Baseline) BART-base T5-base (Baseline) t5-small | BART-base t5-small ByT5-small (Baseline) T5-base (Baseline) mT5_small | mT5_small t5-small mT5_base BART-base mT5_large | t5-small mT5_small BART-base mT5_large mT5_base | ByT5-small (Baseline) ByT5-base (Baseline) BART-base mT5_small ByT5-large (Baseline) | t5-small mT5_large mT5_small mT5_base BART-base | t5-small mT5_small mT5_large mT5_base BART-base | t5-small BART-base mT5_base mT5_large mT5_small | t5-small BART-base mT5_small mT5_base mT5_large | t5-small mT5_base mT5_large mT5_small BART-base | t5-small BART-base mT5_small mT5_base mT5_large | t5-small mT5_large mT5_base mT5_small BART-base | t5-small mT5_large mT5_base BART-base mT5_small | t5-small mT5_base mT5_large mT5_small BART-base | t5-small mT5_base mT5_large BART-base mT5_small | t5-small BART-base mT5_base mT5_large mT5_small | t5-small BART-base mT5_base mT5_large mT5_small |
| wiki auto asset turk test turk | SimpleNER t5-small mT5_base mT5_large T5-xl (Baseline) | t5-small mT5_base mT5_large mT5_small BART-base | t5-small mT5_large mT5_base mT5_small T5-xl (Baseline) | SimpleNER t5-small SimpleNER_baseline mT5_large mT5_xl | BART-base ByT5-small (Baseline) t5-small T5-small (Baseline) ByT5-base (Baseline) | BART-base T5-small (Baseline) t5-small mT5_small ByT5-small (Baseline) | t5-small mT5_small BART-base mT5_large mT5_base | t5-small mT5_small mT5_large mT5_base BART-base | ByT5-small (Baseline) ByT5-base (Baseline) T5-base (Baseline) BART-base ByT5-large (Baseline) | t5-small BART-base mT5_base mT5_large mT5_small | t5-small mT5_large mT5_base mT5_small mT5_xl | t5-small BART-base mT5_large mT5_base mT5_small | t5-small BART-base mT5_large mT5_small mT5_base | t5-small BART-base mT5_large mT5_base mT5_small | t5-small BART-base mT5_small mT5_large mT5_base | t5-small BART-base mT5_large mT5_base mT5_small | t5-small BART-base mT5_large mT5_base mT5_small | t5-small BART-base mT5_large mT5_base mT5_small | t5-small BART-base mT5_base mT5_large mT5_small | t5-small BART-base mT5_large mT5_base mT5_small | t5-small BART-base mT5_large mT5_base mT5_xl |